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FAA  Strategies  for  Reducing  Operational  Error  Causal  Factors 


Introduction 

The  US  Federal  Aviation  Administration  (FAA) 
oversees  the  largest,  safest,  and  most  complex  aviation 
vsystem  in  the  world,  relying  on  a  workforce  of  highly 
trained  air  traffic  control  specialists  who  interact  with 
an  environment  of  radar,  computers,  and  communica¬ 
tion  facilities  to  maintain  the  safety  and  efficiency  of 
the  system.  In  fiscal  year  (FY)  2000  alone,  the  US  air 
traffic  system  handled  166,669,557  operations.  Calcu¬ 
lated  as  a  percent  of  facility  activities,  the  operational 
error  (OE)  rate  per  100,000  activities  increased  from 
.60  in  calendar  year  (CY)  1999  to  .69  in  CYOO  and  .74 
in  CYOl,  then  declined  by  1 1%  to  .66  in  CY02^  (FAA, 
2003a).  Although  air  traffic  declined  after  the  events  of 
September  1 1 , 200 1 ,  the  OE  rate  reflects  the  continuing 
need  to  identify  mitigation  strategies. 

The  FAA  has  historically  tried  to  understand  and 
prevent  the  incidence  of  OEs.  To  accomplish  this,  an 
elaborate  and  detailed  incident  reporting  system  evolved 
to  capture  causal  factors  related  to  their  occurrence.  The 
data  were  intended  to  supply  information  about  where 
to  target  intervention  strategies,  with  several  initiatives 
focusing  on  one  critical  component  of  the  system — the  air 
traffic  control  specialist  (ATCS).  Because  the  controller  is 
the  closest  person  to  the  air  traffic  situation  and  the  last 
point  of  prevention,  much  attention  was  logically  focused 
on  controller  performance.  Indeed,  a  controller  s  action  or 
inaction  provides  a  channel  through  which  pre-existing 
system  vulnerabilities  can  be  manifested.  As  Fisher  (2002) 
noted,  human  error  is  the  mechanism  that  translates 
this  potential  for  making  a  mistake  into  an  occurrence. 
Thus,  performance  of  the  controller  will  always  be  at 
the  sharp  end  of  the  operational  system  (Dekker,  2002) 
in  the  complex  and  multifaceted  environment  of  ATC. 
Whereas  past  initiatives  focused  on  remedial  training  and 
targeted  deficient  performance,  several  recent  initiatives 
have  focused  on  human  performance  within,  and  as  it 
interacts  with,  the  larger  ATC  system,  and  viewed  the 
human  element  as  a  fundamental  part  of  this  complex 
environment. 


Initiatives 

In  recent  years,  the  FAA  Air  Traffic  Evaluations  and 
Investigations  StafFbegan  several  programs  that  focused 
more  attention  on  skill  building  and  performance  mainte¬ 
nance  rather  than  on  remedial  training.  This  approach  is 
based  on  the  philosophy  of  adult  education  and  individual 
responsibili tyfor  maintenance  ofbest  performance  rather 
than  viewing  training  from  a  directive  “schoolhouse”  ap¬ 
proach,  often  disparagingly  referred  to  as  the  “blame  and 
train”  method.  Initiatives  included  fielding  automation 
to  re-create  traffic  situations,  developing  safety  metrics, 
analyzing  incident  data  to  identify  performance  enhance¬ 
ment  opportunities,  and  sponsoring  research  to  further 
develop  the  capability  to  identify  causal  factors. 

Incident  Re-creation 

As  computer  capabilities  increased,  the  idea  that 
computer  processing  could  be  harnessed  to  re-create 
operational  errors  became  a  reality.  During  the  1990s, 
the  FAA  Civil  Aerospace  Medical  Institute  (CAMI) 
collaborated  ^vith  the  FAA  Air  Traffic  Evaluations  and 
Investigations  staff  and  Atlanta  air  route  traffic  control 
center  (ARTCC)  to  develop  automation  to  graphically 
re-create  radar  data  that  were  routinely  recorded  by  en 
route  air  traffic  control  facilities. 

Referred  to  as  the  Systematic  Air  Traffic  Operations 
Research  Initiative  (SATORl),  it  was  developed,  tested, 
and  fielded  to  all  en  route  facilities  and  regional  quality 
assurance  offices  v^fith  the  goal  of  gaining  “a  better  under¬ 
standing  of  the  interaction  between  the  various  elements 
of  displayed  information,  verbal  interactions,  and  the 
control  actions  taken  by  air  traffic  control  specialists” 
(Rodgers  &  Duke,  1993,  pg.  1).  SATORl  is  still  currently 
in  operational  use  and  enables  its  users  to  re-create  seg¬ 
ments  of  operational  traffic  in  a  format  similar  to  what 
was  displayed  to  the  ATCS,  for  example,  showing  relative 
location  and  separation,  speeds,  and  headings  of  aircraft. 
Among  other  things,  SATORl  can  display  full  and  limited 
data  blocks,  beacon  targets,  and  conflict  alerts.  Video  and 
audio  are  synchronized,  and  the  air  traffic  situation  can 
be  displayed  in  four  dimensions. 


^Calculations  of  rates  use  1 5  decimal  places  but  are  rounded  to  2  places 
for  the  table  on  page  6  of  the  FAA  Administrator  s  Fact  Book. 
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Figure  1.  SATORI  system  in  use. 


At  en  route  facilities,  SATORI  systems  enable  the 
facility  quality  assurance  staff  to  re-create  OE  situations 
for  the  controllers  involved  (see  Figure  1). 

It  is  important  to  note  that  the  system  was  not  in¬ 
tended  to  be  used  to  “call”  an  OE;  that  is,  to  identify 
when  an  OE  occurred.  An  OE  was  first  determined  to 
have  occurred  and  then  SATORI  was  used  to  review  the 
situation.  Systems  located  at  regional  and  headquarters 
offices  enable  further  incidentreview.  FAAOrder72 10.56 
provides  guidance  for  the  use  of  all  replay  tools. 

As  technological  advances  aie  made,  the  ATC  system  must 
adjust  to  these  changes  and  ensure  that  radar  reduction  tools 
are  used  correcdy  and  consistently  throughout  the  s)"stem 
in  order  to  provide  the  most  accurate  re-creation  possible 
(FAA  Order  7210.56,  pg.  1-4). 

After  an  OE  is  identified  (or  “called”),  SATORI  re¬ 
creations  are  useful  in  determining  aspects  of  controller 
and/or  pilot  performance  involved  in  the  event.  Re-cre¬ 
ations  can  also  be  used  to  review  peak  periods  of  traffic 


^  Technical  performance  issues  consist  of  areas  of  knowledge  and 
application  that  might  benefit  from  training.  These  issues  are  not 
necessarily  areas  of  deficiency.  An  employee  may  demonstrate  overall 
acceptable  technical  proficiency  but  might  benefit  from  technical 
training  in  the  application  of  a  particular  skill  or  task  (FAA  Order 
7210.56,  pg.  3-1). 


flow  (“pushes”)  and  the  effects  of  weather  on  traffic  flow. 
Viewing  the  re-creation  also  helps  to  target  specific  sidll 
enhancement  programs  for  those  employees  involved  in 
the  event.  In  addition  to  helping  with  the  identification 
of  performance  issues,  re-creations  of  randomly  selected 
traffic  samples  not  related  to  OE  situations  have  also 
been  productively  used  to  assess  controllers^  technical 
proficiency  in  relation  to  training.^ 

The  first  of  its  kind  to  harness  the  capability  of  comput¬ 
ers  to  re-create  air  traffic  situations,  SATORI  re-creations 
have  also  been  used  to  provide  assistance  to  other  agen¬ 
cies’  investigations  of  incidents  involving  aircraft.  These 
have  included  the  National  Transportation  Safety  Board 
(NTSB),  US  Department  of  Justice,  and  the  National 
Aeronautics  and  Space  Administration  (NASA). 

In  addition  to  performance  management  and  investi¬ 
gation,  re-creations  of  traffic  samples  not  related  to  OEs 
have  been  used  in  human  factors  research  at  CAML 
For  example,  traffic  samples  were  used  to  study  sector 
complexity,  controller  workload,  and  performance  (Man¬ 
ning  et  al.,  2001;  Mills,  Pfleiderer,  &  Manning,  2002). 
TRACON  SATORI,  a  prototype  system  for  the  terminal 
environment,  was  used  to  examine  how  controllers  use  in¬ 
formation  about  aircraft  relative  position  to  maintain  “the 
picture”  of  the  traffic  situation  (Pounds,  in  review). 
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Used  effectively,  the  capability  to  re-create  traffic  situ¬ 
ations  can  bring  about  beneficial  changes  in  procedures, 
airspace,  and  future  ATC  systems.  SATORI  re-creations 
are  currently  being  used  in  diverse  ways  to  enhance  sys¬ 
tem  performance.  It  is  anticipated  that  next  generation 
re-creation  tools  currently  being  developed  will  continue 
to  provide  added  value. 

Calibrating  Incident  Set^erity 

The  FAA  Evaluations  and  Investigations  Staff  moni¬ 
tors  the  frequency  of  operational  errors  to  determine  the 
vsystem  vulnerabilities  contributing  to  each  incident  so 
that  they  can  be  identified  and  reduced.  Once  a  relevant 
separation  standard  is  violated  by  ATC,  an  OE  is  re¬ 
corded.  Every  violation  of  separation  standards  provides 
an  important  opportunity  for  lessons  learned  and  system 
improvement,  although  not  all  operational  errors  share  the 
same  characteristics.  Separation  standards  and  procedures 
differ  depending  upon,  for  example,  the  type  of  airspace, 
weather  conditions,  type  of  aircraft,  and  altitude. 

A  study  conducted  by  Rodgers  and  Nye  (1993)  in¬ 
vestigated  whether  the  number  of  aircraft  being  worked 
by  the  controller  or  the  traffic  complexity  at  the  time 
of  the  OE  was  related  to  its  severity,  defined  in  terms 
of  vertical  and  horizontal  separation  between  aircraft. 
Three  categories  were  created  by  assigning  a  maximum 
often  possible  points  each  to  the  horizontal  and  vertical 
separation  reported.^  Based  on  the  total  point  value,  OEs 
were  partitioned  into  categories  of  major  (20  points), 
moderate  (14-19  points),  and  minor  (13  or  less  points) 
severity.'^ 

Results  of  the  Rodgers  and  Nye  study  were  counterin¬ 
tuitive  and  demonstrated  that  neither  number  of  aircraft 
nor  traffic  complexity  was  significantly  related  to  major, 
moderate  or  minor  OE  severity,  although  the  analyses 
revealed  ways  that  the  reporting  process,  and  thus  the 
resultant  data,  could  be  improved.  The  authors  recom¬ 
mended  gathering  more  normative  rather  than  descriptive 
data,  increasing  the  reliability  of  the  reported  data,  and 
using  a  re-creation  capability  to  permit  investigators  to 
review  the  dynamics  of  the  air  traffic  situation. 

The  U.S.  Department  of  Transportation,  Office  of 
Inspector  General  (December  2000)  recommended  that 
the  FAA  Air  Traffic  Investigations  Division  tackle  the 
problem  of  modeling  and  defining  severity  for  OEs  in 
flight  to  describe  the  degree  that  the  applicable  separa¬ 
tion  standard  was  violated.  The  purpose  was  to  group 
airborne  OEs  as  low,  moderate,  or  high  severity  and  thus 


^  Veriical  separaiion  was  subdivided  depending  upon  whether  the 
incident  occurred  below  or  above  29,000  feet  (FL290). 

Of  the  1053  OEs  in  their  sample,  only  15  (.01%)  were  coded  as 
“major”  severity. 


be  able  to  focus  resources  on  the  most  severe  events  and 
to  identify  factors  related  to  specific  categories  of  events. 
Data  about  systemic  causes  of  OEs  could  then  be  used 
to  more  explicitly  direct  action  towards  prevention  of 
fiiture  occurrences. 

The  FAA  Air  Traffic  Evaluations  and  Investigations  Staff 
developed  a  classification  system  to  distinguish  between 
OEs  on  these  dimensions.  Classification  categories  were 
developed  to  reflect  the  operational  environment.  In  the 
first  version  of  the  index,  an  OE  was  classified  on  each 
dimension  according  to  its  severity — the  extent  to  which 
separation  distance  was  reduced — as  low  (39  points  and 
below),  moderate  (40-89  points),  or  high  (90  points 
and  above)  using  a  100-point  scale.  Objective  distances 
were  used  to  minimize  subjective  interpretations  of  the 
data.  Actual  radar  data  from  numerous  operational  errors 
representing  en  route  facilities  nationwide  were  used  to 
test  the  adequacy  of  the  categories  and  the  classification 
methodology.  The  components  of  the  model  included 
elements  associated  with  loss  of  standard  separation,  such 
as  the  relationship  of  the  aircraft  in  conflict  to  one  another 
(e.g.,  converging  versus  diverging  courses),  closure  rate, 
and  level  of  ATC  involvement  -  whether  the  event  was 
a  ‘controlled”  or  “uncontrolled”  OE.  The  point  distri¬ 
butions  for  en  route  radar  OEs  are  shown  in  Table  1.  A 
similar  table  was  developed  for  OEs  in  terminal  and  en 
route  airspace  with  single-site  radar. 

The  initial  step  in  calculating  error  severity  was  to 
determine  the  lateral  and  vertical  proximity  between  the 
involved  aircraft.  The  horizontal  and  vertical  distances 
were  defined  as  the  minimum  separation  based  on  the  radar 
data  j  ust  prior  to  aircraft  divergence.  Situations  with  faster 
aircraft  closure  rates,  coupled  with  converging,  opposite 
direction  flight  paths  were  assumed  to  present  a  greater 
threat  than  slower  closure  rates  and  diverging  flight  tracks. 
The  idea  was  that  head-on  encounters  with  high  rates 
of  speed,  coupled  with  minimum  radar  data  separation 
distance  prior  to  divergence  should  account  for  the  greatest 
total  point  value  in  the  model.  An  assumption  was  made 
that  aircraft  not  crossing  each  others  paths  greatly  reduces 
the  threat  to  safety,  and  consequendy,  does  not  receive 
severity  index  points  (see  Table  1).  In  addition,  if  aircraft 
are  not  converging,  closure  rates  become  less  significant 
when  the  other  parameters  are  properly  factored. 

This  model  does  not  imply  in  any  way  that  any  mini¬ 
mum  separation  less  than  those  required  by  FAA  Order 
7110.65  is  acceptable  from  an  operational  perspective. 
The  main  purpose  of  the  ATC  system  is  to  preserve  safety. 
To  learn  from  the  occasional  shortfall  in  the  ATC  system, 
the  controllers  action  prior  to  the  OE  was  also  included 
in  the  assessment  of  the  total  severity  of  an  event.  A  “con¬ 
trolled”  OE  was  defined  as  an  OE  where  the  controller 
was  aware  of  the  impending  conflict  and  took  corrective 
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Table  1 .  Radar  OE  Severity  Index  terminal  and  en  route  single-site  chart  (FAA  Order  7210.56) 


VERTICAL  SEPARATION 

POINTS 

HORIZONTAL  SEPARATION* 
3-mile  separation  requirement  •  T 

:^:MNTS:;:::i 

Less  than  500  feet 

25 

Less  than  V2  mile 

25 

500  feet  to  599  feet 

V2  mile  to  0.999  mile 

18 

600  feet  to  699  feet 

16 

1  mile  to  1.499  miles 

14 

700  feet  to  799  feet 

12 

1.5  miles  to  2  miles 

800  feet  to  899  feet 

6 

2  miles  to  2.499  miles 

6 

900  feet  to  999  feet 

2 

2.5  miles  to  2.999  miles 

2 

CLOSURE  RATE 

POINTS 

HORIZONTAL  SEPARATION 
2.5-mile  f^uirement 

POINTS 

700  knots  and  greater 

10 

Less  than  Vi  mile 

25 

300  knots  to  699  knots 

8 

Vi  mile  to  0.999  mile 

20 

100  knots  to  299  knots 

6 

1  mile  to  1.499  miles 

16 

Less  than  100  knots 

4 

1.5  miles  to  1.999  miles 

10 

FLIGHT  PATHS 

POINTS 

2  miles  to  2.499  miles 

4 

Converging  -  Opposite  Courses 

20 

ATC  CONTROL  FACTOR 

POINTS 

Converging  -  Crossing  Course 

18 

Uncontrolled 

20 

Same  Course 

10 

Controlled  with  TCAS  RA 

15 

0 

Controlled  with  no  TCAS  RA 

4 

*  When  wake  turbulence  separation  standards  are  governing,  DO  NOT  include  any  vertical  point  value. 
Instead  use  the  appropriate  in  trail  separation  index  below,  as  well  as  other  applicable  factors. _ 


action  to  increase  separation.  An  “uncontrolled”  OE  was 
defined  as  an  OE  where  the  controller  was  unaware  of 
the  conflict,  took  no  corrective  action,  and/or  became 
aware  of  the  conflict  but  did  not  have  enough  time  to 
effectively  mitigate  the  loss  of  separation. 

The  severity  model  assigns  a  total  of  100  points.  Com¬ 
ponent  categories  are  used  to  allot  point  values  correspond¬ 
ing  with  their  relative  significance  d  uring  the  event.  Table 
1  shows  the  distribution  of  points  possible  for  terminal 
and  en  route  facilities  with  single  site  radar.  For  example, 
horizontal  separation  of  less  than  one-half  mile  in  airspace 
requiring  at  least  3  miles  horizontal  separation  would  be 
assigned  a  point  value  of  25;  closure  rate  of  700  knots 
and  greater  would  be  assigned  10  points.  These  values 
would  be  summed  with  points  from  the  other  appropri¬ 
ate  categories.  The  remainder  of  the  100  points  left  after 
the  assigned  points  have  been  subtracted  is  the  “safety 
factor.”  Figure  2  shows  a  hypothetical  example  of  how 
the  points  in  each  category  of  the  severity  index  can  be 
used  graphically  to  display  the  various  elements  of  one 
OE.  A  different  table  is  used  for  en  route  OEs  requiring 
5'miles'  separation. 

This  severity  index  is  an  algorithmic  approach,  based 
on  objectively  measurable  and  observable  variables  in 
the  operational  environment.  As  the  method  was  further 
developed,  the  initial  three  categories  were  split  into  four: 


High  Severity  (A);  Moderate  Severity,  uncontrolled  (B); 
Moderate  Severity,  controlled  (C);  and  Low  Severity  (D). 
The  four  category  model  made  it  conceptually  similar  to, 
and  potentially  confusable  with,  less  objective  methods 
being  used  by  other  groups  to  classify  incident  types  such 
as  runway  incursions. 

Datd'drwen  Focus  on  Performance 

Analysis  of  OE  data  by  the  FAA  Office  of  Investiga¬ 
tions  revealed  several  recurring  causal  factors,  for  example, 
readback/hearback  errors  and  position  relief  briefings.  As 
a  result,  interventions  were  initiated  to  address  them.  A 
series  of  videos  was  also  produced  to  communicate  several 
types  of  complex  system  vulnerabilities. 

Readback/hearback  errors.  Readback/hearback  errors 
occur  if  the  pilot  incorrectly  repeats  back  to  the  control¬ 
ler  the  instruction  or  information  just  received  from  the 
controller  and  if  the  controller  fails  to  catch  the  pilots 
incorrect  response.  As  an  example,  a  readback/hearback 
error  would  be  noted  when  the  controller  instructs  “Piper 
123  climb  and  maintain  1-0  thousand,”  but  the  pilot 
repeats  back  to  the  controller  “Roger.  Piper  123  climb¬ 
ing  to  1  -  2  thousand.”  If  the  controller  does  not  catch 
the  pilot  s  error  in  the  response,  it  would  be  classified  as 
a  readback/hearback  error.  Although  these  are  human 
errors,  they  do  not  necessarily  lead  to  an  OE. 
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Figure  2.  Sample  of  a  moderate  severity  OE,  with  the  percent  of  total  points 
for  each  element  shown. 


Awareness  programs  targeting  readback/hearback  er¬ 
rors  were  developed  by  headquarters  staff  as  well  as  by 
regional  and  facility  groups.  Facility  programs  included 
‘‘tape  talks”  where  voice  recordings  for  the  ATCS  on 
a  position  were  reviewed  by  the  ATCS  and/or  facility 
staff  specialists  to  assess  communication  performance. 
The  Air  Traffic  Investigations  Division  Staff  produced  a 
video  —  “Preventing  Readback  Errors”  -  highlighting 
how  different  influences,  such  as  ambient  noise  and  dis¬ 
traction,  can  contribute  to  the  occurrence  of  this  type  of 
error.  The  video  was  sent  to  all  facilities  as  a  mandatory 
briefing  item.  Based  on  its  initial  success,  this  awareness 
program  became  an  annual  event  with  January  inaugu¬ 
rated  as  Hearhack  -  Readback  Awareness  Months  which 
included  constant  emphasis  on  good  communications 
skills,  random  tape  monitoring  to  highlight  examples  of 
correct  phraseology,  and  positive  coaching. 

Position  relief  briefings.  Position  relief  briefings  take 
place  when  one  controller  assumes  (takes  over)  respon¬ 
sibility  for  a  position  from  another  controller,  transfer¬ 
ring  responsibility  from  one  controller  to  the  other.  The 
position  relief  briefing  is  a  standard  operating  process 
designed  to  optimize  transfer  of  responsibility  while  at 
the  same  time  minimizing  the  additional  workload  as¬ 
sociated  with  the  task  of  transferring  duty.  The  relieving 
controller  previews  the  position  and  then  indicates  to 
the  controller  being  relieved  that  the  verbal  briefing  may 


begin.  A  checklist  covers  items  to  be  noted  prior  to  the 
relieving  controller  assuming  responsibility  for  the  posi¬ 
tion.  The  relieving  controller  observes  the  position  and 
then  the  controller  being  relieved  points  out  any  abnormal 
items,  traffic  situation,  and  any  other  issues  of  concern 
using  the  checklist.  Thorough  coverage  of  the  checklist 
items  is  meant  to  ensure  that  the  relieving  controller 
“has  the  picture”  of  the  situation.  That  is,  the  controller 
being  relieved  ensures  that  the  relieving  controller  sees 
all  relevant  information,  understands  the  situation,  and 
is  aware  of  any  potential  conflicts  or  problems.  After  the 
relieving  controller  assumes  responsibility  for  the  posi¬ 
tion,  the  controller  being  relieved  observes  the  overall 
position  operation  to  determine  if  assistance  is  needed 
and  provide  or  summon  it  as  appropriate. 

Trends  in  the  OE  data  showed  that  OEs  frequently 
occurred  within  10  minutes  of  a  controller  taking  over  a 
position.  Figure  3  shows  the  distribution  of  1,056  OEs 
between  1997  and  2000,  and  the  number  of  OEs  that 
occurred  within  5>  10,  and  20  minutes  of  the  controller 
taking  over  a  position.  As  a  percent  of  the  total  number 
of  OEs  (Figure  4),  approximately  9%  occurred  within 
the  first  5  minutes,  18%  within  the  first  10  minutes,  and 
35%  within  the  first  20  minutes  on  position. 

An  initiative  to  address  this  problem  required  all 
managers  to  validate  position  relief  checklists,  as  well 
as  provide  a  capability  for  recording  the  briefing.  Shift 
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Figure  3.  Number  of  1056  OEs  occurring  relative  to  the  controller’s  time  on 
position  and  workload  (FY  1997  through  FY  2000). 


Figure  4.  Percent  of  1056  OEs  occurring  relative  to  the  controller’s  time  on 
position  (FY  1997  through  FY  2000  from  Figure  3). 


supervisors  were  required  to  ensure  the  use  of  a  position  remain  at  the  position.  Other  facilities  adopted  this  as 

relief  briefing  checklist  and,  where  available,  ensure  that  a  good  practice  and  permit  the  overlap  period  to  vary 

position  relief  briefings  were  recorded.  Controllers  were  depending  upon  traffic  demands.  The  intent  in  this  lat- 

trained  and  encouraged  to  accept  position  responsibility  ter  practice  is  that  basing  the  overlap  period  on  traffic 

only  after  they  were  fully  aware  of  the  traffic,  and  the  demand  reduces  the  likelihood  of  potential  distractions 

relieving  controller  and  the  controller  being  relieved  were  associated  with  multiple  personnel  remaining  on  the  same 

to  establish  an  appropriate  overlap  period  to  complete  operational  position  unnecessarily, 

the  transfer  of  responsibilities.  Some  facilities  mandate  a  Video  brii^ng  materials.  The  Air  Traffic  Investigations 
specific  amount  of  time  that  the  relieved  controller  shall  Division  Staff  produced  four  videotapes  to  focus  awareness 
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on  different  types  of  causal  factors  found  to  be  related 
to  OEs.  ^  Copies  of  these  were  sent  to  all  FAA  air  traffic 
facilities  to  be  used  for  briefing  materials. 

Accidents  often  occur  after  a  series  of  inconsequential 
events  that  create  links  in  a  chain  during  which  individu¬ 
als  have  had  a  number  of  opportunities  to  intercede  and 
break  the  chain.  The  “Break  the  Chain”  video  illustrates 
how  events,  if  uninterrupted,  can  culminate  in  an  accident 
and  how  attending  to  details  can  help  break  the  chain  of 
events  and  prevent  accidents  and  incidents.  The  episode 
portrayed  in  the  video  is  fictional  but  was  compiled  from 
actual  events.  A  small  business-owned  aircraft  is  carrying 
a  group  of  business  executives  to  a  meeting.  The  flight 
departs  later  than  expected  due  to  the  passengers*  late 
arrival  and  delays  in  air  traffic  services.  The  original  plan 
was  to  depart  ahead  of  developing  stormy  weather;  how¬ 
ever,  the  delay  and  the  quickly  developing  storm  result 
in  moderate  icing  conditions.  An  aircraft  mechanic,  the 
passengers,  both  pilots,  an  airway  facility  staff  specialist, 
and  multiple  air  traffic  control  specialists  (both  terminal 
and  en  route)  all  either  contributed  in  some  way  to  the 
chain  of  events  or  failed  to  interrupt  the  sequence  when 
they  had  the  opportunity. 

“Collision  Course:  What  are  the  odds?”  depicts  an 
actual  event,  demonstrating  how  rare  and  improbable 
events  can  and  do  occur.  Individually,  each  event  would 
be  fairly  benign.  However,  as  part  of  a  chain  of  events, 
seemingly  inconsequential  errors  can  set  in  motion  a  series 
of  events  that  cannot  be  undone.  Outwardly  insignificant 
errors  committed,  for  the  most  parr,  by  controllers  in  two 
air  route  traffic  control  centers,  put  two  large  jet  aircraft 
on  a  converging  course  and  that  information  could  not 
be  relayed  from  the  ground  to  the  aircraft  because  radio 
communications  were  lost.  The  video  demonstrates 
how  the  failure  to  accomplish  routine  procedures  such 
as  switching  an  aircraft  to  another  frequency,  becoming 
momentarily  distracted,  or  being  complacent  about  an 
evolving  situation  can  potentially  result  in  large  and  un¬ 
expected  adverse  outcomes,  although  a  midair  collision 
was  avoided  in  the  event  depicted. 

“Consequences  of  Simple  Omissions”  is  a  compilation 
of  actual  events.  This  video  illustrates  examples  of  how 
small  omitted  actions,  lack  of  attention,  failure  to  fol¬ 
low  operational  practices,  compounded  by  poor  facility 
practices,  and  lack  of  self  discipline  or  professionalism 
resulted  in  incidents  ranging  from  operational  errors  to 
fatal  accidents.  The  video  advocates  adhering  to  indi¬ 
vidual  professionalism  and  remembering  the  importance 
of  maintaining  standards  and  accuracy  for  safety. 


^  Ms.  Christine  Soucy,  a  safety  investigator  with  AAT-200  wrote  and 
helped  direct  the  videos  for  this  program. 


The  video  titled  “Preventing  Readback  Errors,”  men¬ 
tioned  earlier,  focused  on  communication  strategies  to 
reduce  misunderstandings  between  controllers  and  pi¬ 
lots.  Strategies  include  reducing  the  complexity  of  each 
communication  and  chunking  information  within  the 
communication  to  facilitate  understanding.  Information 
is  presented  in  a  humorous  manner  to  illustrate  strategies 
to  overcome  common  blocks  to  good  communication. 

Identifying  OE  Causal  Factors 

The  need  for  a  formal  reporting  process  was  recognized 
early  in  the  FAAs  evolution  and  by  1965,  FAA  Order 
8020  had  established  the  ATS  System  Error  Reporting 
Program.  Early  on,  recommendations  were  developed 
regarding  the  conduct  of  incident  investigations  and  the 
use  of  the  resulting  information,  many  of  which  were 
incorporated  into  the  Order  (O’Connor  &  Pearson, 
1965).  For  instance,  O’Connor  and  Pearson  suggested 
that  any  reporting  system  should  be  based  on  a  system 
view,  including  controller  performance,  the  influence  of 
personal  capacities  and  skills,  the  design  and  operation 
of  the  system,  and  modifiers  of  the  working  environment 
such  assupervision,  operating  procedures,  health,  morale, 
and  work  schedules.  Notably,  recommendations  made 
by  O’Connor  and  Pearson  remain  as  relevant  today  as 
they  were  then. 

Air  traffic  control  has  become  an  increasingly  complex 
system  involving  men  and  equipment  in  a  continuous  and 
dynamic  decision-making  fuunction.  Future  developments 
point  to  the  rising  use  of  complex  equipment,  inclitding 
high-speed  computers,  as  aircraft  speeds  and  the  system's 
load  continue  to  rise.  By  projecting  current  trends,  it  can 
be  anticipated  that  future  system  changes  involving  equip¬ 
ment,  personnel,  and/or  procedures  point  to  the  need  for 
longer  and  longer  lead  times  as  complexity  grov's.  The 
above  considerations  point  to  the  need  for  close  scrutiny 
of  the  ongoing  system  failures  and/or  incidents  in  order  to 
provide  the  most  accurate  feedback  information  for  system 
correction  or  modification  (pg  1). 

Additionally,  O’Connor  and  Pearson  recommended 
an  approach  to  system  error  evaluations,  asserting  that 
“the  man -machine  system  Avill  never  be  perfectly  reliable, 
efficient,  and  error  free  because  of  the  inherent  limita¬ 
tions  and  idiosyncrasies  of  the  human  component”  (pg. 
1)  and  that  a  deliberate  and  objective  process  would  be 
better  than  “shooting  in  the  dark  for  sources  of  error  and 
possible  solutions”  (pg.  1). 

Later,  other  changes  were  periodically  made  to  the 
incident  reporting  form.  For  example,  in  1997  a  version 
was  fielded  having  three  additional  causal  factors  under 
the  category  of  Inappropriate  use  of  Displayed  Data,  This 
category  was  associated  with  the  controllers  use  of  the 
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radar  display  and  situation  awareness  associated  with 
use  of  the  radar  data.  The  three  added  items— failure  to 
detect  displayed  data,  failure  to  comprehend  displayed  data, 
znA  failure  to  project future  status  of  displayed  data — ^were 
included  to  address  the  controllers  situation  awareness. 
This  change  proved  initially  successful,  with  subsequent 
OE  data  showing  that  use  of  the  Inappropriate  use  of 
Displayed  Data  -  Other  category  decreased  and  the  data 
that  would  otherwise  have  been  attributed  to  this  Other 
category  distributed  across  these  three  new  categories 
(Rodgers,  personal  communication).  Although  this 
change  brought  finer  detail  to  the  description  of  the 
OE,  the  issue  of  reporting  reliability  raised  by  Rodgers 
and  Nye  (1993)  was  still  an  issue. 

Currently,  the  FAA  Air  Traffic  Investigations  Division 
oversees  and  coordinates  the  OE  reporting  process  gov¬ 
erned  by  the  FAA  Air  Traffic  Quality  Assurance  Order 
7210.56  (FAA,  2002).The  order  “provides  specific  guid¬ 
ance  on  investigation,  reporting,  and  recording  types  of 
incidents  that  impact  the  quality  of  air  traffic  services” 
(pg.  i).  Specifically,  section  5'1'2  (pg.  5-2)  stipulates: 

5-1-2.  SUSPECTED  EVENT 

a.  In  order  to  mainrain  an  effective  Air  Traffic  System, 
it  is  imperative  that  we  identify  all  deficiencies  within  our 
system  and  take  appropriate  corrective  aaions  necessary  to 
fix  any  associated  problems.  Operational  errors  and  devia¬ 
tions  are  reported  for  just  that  reason,  so  those  problems 
(either  systemic  or  individual)  can  be  corrected  to  enhance 
system  integrity.  The  identific.ation  of  operational  errors 
and  deviations  without  fear  of  reprisal  is  an  absolute  re¬ 
quirement  and  is  the  responsibility  of  all  of  us  who  work 
within  our  system. 

b.  Accordingly,  it  remains  Air  Traffic  Policy  tliat  any 
employee  who  is  aware  of  any  occurrence  that  may  be  an 
operational  error,  deviation,  orair traffic  incident  (as  defined 
in  paragraph  4-1-1,  Definitions),  immeciiatcly  report  the 
occurrence  to  any  available  supervisor,  controller-in-cliarge 
(CIC)  or  management  official. 

To  develop  information  so  that  data-driven  decisions 
about  causal  factors  and  intervention  strategies  could 
be  made,  the  Office  of  Investigations  determined  that  a 
method  for  identifying  causal  factors  related  to  human 
performance  was  needed — ^a  method  that  viewed  human 
performance  as  one  among  several  potential  points  of 
system  vulnerability.  Resulting  information  about  human 
factors  could  then  be  proactively  used  to  mitigate  the 
potential  for  future  incidents.  This  effort  was  responsive 
to  goals  in  the  FAAs  1999  Strategic  Plan,  including  the 
goal  to  “eliminate  accidents  and  incidents  caused  by 
human  error”  (FAA,  1999).  The  FAAs  National  Avia¬ 
tion  Research  Plan  for  1999  also  echoed  the  intended 
outcome  of  developing  enhanced  measures  of  human 
performance  and  increased  understanding  of  factors  that 
lead  to  performance  decrements. 


An  initial  effort  was  undertaken  in  coordination  with 
the  FAAs  Civil  Aerospace  Medical  Institute  (CAMI)  to 
determine  whether  retrospective  analysis  of  existing 
OE  reports  could  extract  additional  useful  information 
by  using  a  human  factors  approach  (Pounds  &  Isaac, 
2001;  Pounds  &  Scarborough,  2002).  The  retrospective 
analyses  relied  on  data  from  existing  OE  reports  that 
were  based  on  standardized  procedures  specified  by  FAA 
Order  72 1 0.56.  Outcomes  from  this  work  suggested  that 
standard  reporting  procedures  did  not  require  facilities  to 
collect  the  type  of  data  necessary  to  perform  a  thorough 
human  factors  analysis.  That  is,  although  the  report  forms 
captured  descriptive  data  about  the  OE,  little  information 
was  collected  about  events  and  causal  factors  preceding 
and  during  the  loss  of  separation. 

As  these  retrospective  analyses  were  being  completed, 
the  FAA  Office  of  Aviation  Research  entered  into  a  col¬ 
laborative  agreement  with  Eurocontrol  and  signed  Ac¬ 
tion  Plan  12  (API 2):  Management  and  Reduction  of 
Human  Error  in  ATM.  The  initial  goal  of  this  project 
was  to  examine  two  existing  techniques  for  identifying 
human  error  and  to  determine  whether  they  could  be 
harmonized  into  one  technique.^  If  so,  the  FAA  and 
Eurocontrol  member  states  would  be  able  to  use  the 
technique  retrospectively  to  examine  existing  incident 
reports  for  information  related  to  human  factors  trends 
in  the  data,  and  to  leverage  this  information  to  develop 
and  share  mitigation  strategies. 

As  API 2  activities  progressed,  ATC  subject  matter 
experts  from  both  FAA  and  Eurocontrol  judged  that  the 
harmonized  technique- JANUS  -  also  showed  promise 
as  a  supplement  to  existing  reporting  processes  (Pounds 
&  Isaac,  2001).  That  is,  rather  than  serving  merely  as  a 
retrospective  data  mining  tool,  the  new  technique  might 
also  have  value  if  integrated  directly  into  existing  OE 
reporting  processes.  Based  on  this  hypothesis  and  after 
successful  harmonization  (Pounds  &  Isaac,  2001),  the 
course  of  API  2  was  modified  to  include  further  refine¬ 
ment  and  testing  of  the  technique.  Validation  activities 
posed  unique  challenges  to  both  the  FAA  and  Eurocontrol. 
Based  on  discussions  of  these  differences,  it  was  decided 


^  The  two  techniques  used  for  the  harmonization  were  the  Human 
Factors  Analysis  and  Classification  System  (HFACS;  vShappell  & 
Wiegmann,  2000)  and  the  model  for  Human  Error  in  Air  Traffic 
Management  (HERA;  EATMP,  2003).  The  Human  Factors  Analysis 
and  Classification  System  (HFACS),  a  human  factors  taxonomy  for 
analysis  of  existing  aviation  accident  databases,  was  originally  developed 
for  the  US  Navy  to  investigate  military  aviation  accidents.  HFACS 
is  currently  being  used  by  the  FAA  to  analyze  civil  aviation  accident 
databases.  HERA  is  a  model  of  human  error  for  air  traffic  control 
that  was  developed  as  a  tool  to  increase  the  effectiveness  of  human 
error  identification. 


that  parallel  and  complementary  approachcvS  be  used 
based  on  the  particular  requirements  of  each  to  conduct 
the  validation  activities.  Thus  the  FAA  and  Eurocontrol 
developed  the  technique  to  reflect  their  individual  system 
needs,  resulting  in  two  structurally  parallel  techniques. 

The  FAA  method  is  a  structured  interview  process  that 
leads  the  analyst  through  a  series  of  questions  designed 
to  identify  the  mental  error  and  the  contextual  condi¬ 
tions  surrounding  it.  Questions  related  to  the  controllers 
perceptions,  memory,  and  decision-making  processes  that 
lead  to  execution  of  a  plan  are  included.  The  method  of 
using  the  technique  considers  the  dynamic  ATC  traffic 
environment  and  treats  each  OE  as  a  potential  chain  of 
events  (or  human  errors)  that  result  in  the  final  loss  of 
separation. 

This  line  of  work  expanded  on  efforts  by  the  Air 
Traffic  Evaluations  and  Investigations  Staff  to  improve 
causal  factor  information.  For  an  inexperienced  analyst 
of  an  OE,  the  technique  is  a  potential  aid  to  ask  the  right 
questions  for  eliciting  causal  factors  and  relevant  human 
factors.  For  an  experienced  analyst,  the  technique  ensures 
that  he  or  she  considers  a  broad  range  of  causal  factors 
rather  than  relying  on  experience  and  focusing  on  "the 
usual  suspects.''  Figure  5  illustrates  the  general  categories 
of  contextual  conditions  included  in  the  technique. 


A  research  project  to  test  and  validate  the  JANUS 
technique  was  conducted  in  collaboration  with  the  Air 
Traffic  Evaluations  and  Investigations  Staff  and  the  Na¬ 
tional  Air  Traffic  Controllers  Association.  A  parallel  project 
was  conducted  by  Eurocontrol.  The  research  proposed  to 
answer  several  basic  questions  related  to  validity:  Does 
the  technique  work?  How  well  does  it  work?  Is  it  bet¬ 
ter  than  the  current  method?  Is  it  ready  for  operational 
implementation?  Will  the  results  from  the  technique  help 
to  improve  safety  management? 

Validation  activities  also  posed  unique  challenges  to 
both  the  FAA  and  Eurocontrol.  Based  on  discussions 
of  the  organizational  differences,  it  was  decided  that  a 
harmonized  approach  could  be  defined  that  would  allow 
the  other  organization  to  leverage  the  potentially  comple¬ 
mentary  work,  findings,  and  lessons  learned  while  also 
meeting  the  particular  requirements  of  each  organization. 
Methodologies  for  testing  the  technique  were  adapted 
to  accommodate  the  respective  organizations'  testing 
environments,  including  use  of  operational  resources 
and  test  constraints. 

Results  from  the  validation  activities  conducted  to 
date  by  both  organizations  surest  that  information  pro¬ 
vided  by  the  JANUS  technique  will  help  improve  safety 
management  through  the  more  effective  identification  of 


Figure  5.  FAA  JANUS  taxonomy  showing  the  categories  of  contextual  conditions 
included  in  the  technique. 
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human  factors  associated  with  OEs.  However,  a  definitive 
answer  can  only  be  scientifically  determined  by  a  longitu^ 
dinal  analysis  of  data  gathered  using  this  technique.  The 
information  can  then  be  evaluated  for  its  usefiilness  as 
strategies  are  identified  with  which  to  mitigate  the  po¬ 
tential  for  future  operational  errors.  Indeed,  the  ancient 
Roman  symbol  of  Janus,  after  which  the  harmonized 
technique  was  named,  was  attributed  with  the  ability  to 
look  back  so  that  an  understanding  of  past  events  could 
lead  to  insight  about  future  events. 

Keeping  The  Mental  Edge 

The  National  Air  Traffic  Professionalism  (NATPRO) 
project  is  an  example  of  how  information  identified  by 
OE  analysis  can  be  turned  into  strategy  and  ski  11  enhance¬ 
ment.’'  NATPRO  is  a  new  training  approach  sponsored  by 
the  FAA  Air  Traffic  Investigations  Division.  Rather  than 
relying  solely  on  knowledge-based  training,  this  approach 
integrates  the  concept  of 'performance  coaching,”  using 
an  awareness  seminar  coupled  with  a  practicum. 

Facility  personnel  first  participate  in  the  coaching 
clinic  developed  by  Breedlove  (2003)  that  covers  the 
knowledge  and  critical  skills  of  effective  coaches.  The 
coaches  then  conduct  the  seminar  and  practicum  at  their 
facility.  The  coaches  help  participants  understand  how 
the  seminar  concepts  relate  to  performance  by  pointing 
out  the  connection  between  the  seminar  material  and 
the  practicum  experience  when  participants  practice  the 
activities.  For  example,  the  coach  might  try  to  distract  the 
participant  during  practice  to  demonstrate  the  influence 
of  environmental  distractions  on  performance. 

The  initial  NATPRO  program  is  focused  on  cognitive 
skills  related  primarily  to  visual  attention,  such  as  detec¬ 
tion  of  information,  focusing  on  relevant  information 
and  multitasking.  The  practicum  includes  activities  to 
exercise  the  mind  and  improve  concentration  through 
distributed  practice.  The  basic  concept  is  not  only  lim¬ 
ited  to  concentration.  The  NATPRO  program  concept  is 
designed  to  also  include  other  ATC  skills  such  as  auditory 
attention,  decision  making,  and  planning. 

The  seminar  on  cognitive  skills  was  developed  to  in¬ 
crease  an  individual  s  understanding  of  the  mental  process 
of  concentration,  factors  that  affect  mental  processes,  and 
how  mental  skills  relate  to  performance.  Although  the 
skills  are  generic,  we  hope  that  by  improving  general  skills, 
individuals  will  demonstrate  a  corresponding  improve¬ 
ment  in  controller  performance.  Once  armed  with  the 
knowledge  provided  by  the  seminar,  participants  then 


^  The  NATPRO  project  was  developed  by  Mr,  Randall  Breedlove 
with  assistance  from  Mr.  Jimmy  Mills  (ASO-150)  and  the  Air  Traffic 
Southern  Region  Learning  Council 


experience  their  own  strengths  and  limitations  during 
the  Practicum.  Interactive  web-based  computerized  skill 
challenges  permit  participants  to  gain  insight  about  their 
own  skills.  By  testing  themselves  against  the  computer 
and  experiencing  how  performance  can  vary  in  relation 
to  factors  such  as  distraction,  fatigue,  boredom,  and  so 
on,  participants  gain  increased  understanding  of  their 
own  performance  and  identify  strategies  to  improve  it. 
Putting  the  challenges  in  a  web-based  application  gives 
individuals  and  teams  the  opportunity  to  compete  for 
high  scores,  should  they  desire  to  do  .so.  Although  the 
competitive  aspect  is  available  to  participants,  it  is  not 
required.  It  is  included  to  enhance  the  experience  for 
those  participants  who  want  to  engage  in  interpersonal 
competition. 

Admittedly,  the  successful  transfer  of  skills  from  this 
program  to  actual  air  traffic  control  performance  will  be 
difficult  to  measure  objectively.  There  are  several  measures 
that  maybe  partially  influenced  as  a  result  of  this  training, 
including  a  reduction  in  the  number  of  operational  errors; 
however,  because  the  air  traffic  system  is  complex,  there 
are  many  factors  not  related  to  controller  preparation  and 
awareness  that  also  affect  the  number  of  operational  er¬ 
rors.  Other  less  tangible  measures  may  include  an  increase 
in  efficiency  and  higher  job  satisfaction.  Success  of  the 
program  could  be  captured  by  evaluating  data  from  several 
sources.  Initially,  we  plan  to  compare  the  practicum  skill 
levels  and  subjective  evaluations  from  participants  before 
and  after  participatingin  the  program.  However,  whether 
skill  enhancement  realized  in  the  NATPRO  program  will 
translate  to  working  traffic  cannot  be  objectively  evaluated 
at  this  time.  Measurement  of  performance  using  high-fi¬ 
delity  air  trafficsimulations  or  medium  fidelity  standalone 
simulation  tools  would  supply  more  substantive  evidence 
for  evaluating  the  effectiveness  of  the  program. 

If  this  type  of  program  is  successful,  it  would  demon¬ 
strate  a  more  cost-effective,  personally  rewarding  delivery 
of  training.  The  NATPRO  approach  may  improve  air 
traffic  safety  and  efiiciency  by  increasing  the  controllers 
attention  and  perception  skills,  sustaining  a  highly  skilled 
work  force.  Although  targeting  ATC  performance,  the 
skills  themselves  are  generic.  Participants  would  be  ex¬ 
pected  to  also  benefit  by  transfer  of  skill  to  other  activities 
beyond  their  professional  ATC  work  environment. 

Conclusion 

A  philosophy  of  individ ual  leadership  combined  with 
joint  accountability  guides  these  efforts  and  others,  all 
part  of  the  FAAs  ongoing  efforts  to  maintain  and  enhance 
aviation  safety  by  identifying  system  vulnerabilities  and 
developing  mitigation  strategies.  Air  traffic  control  is  a 
high-demand,  high-consequence  activity,  and  controllers 
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must  rely  on  their  mental  skills  to  successfully  orchestrate 
air  traffic.  The  initiatives  described  here  reflect  a  human^ 
centered  approach  but  situates  the  human  as  the  crux  of 
a  larger,  more  complex,  dynamic  system  of  other  people 
and  computers  in  a  fast-paced  environment  of  radar 
information,  communications,  and  aircraft  movement. 
Although  any  single  initiative  may  not  be  a  solution  in 
itself — no  silver  bullet  that  will  immediately  eliminate  all 
human  error  and  every  operational  error — mitigation  and 
reduction  is  a  journey  of  many  single  steps  that  progress  in 
the  same  direction  toward  prevention  of  these  events. 
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